METHOD AND APPARATUS FOR CROSS-MODALITY COMPARISONS AND 

CORRELATION 

BACKGROUND OF THE INVENTION 

Cross Reference to Related Applications 

[0001] This application claims priority under 35 U.S.C. 119(e) to U.S. Provisional 

Application Serial No. 60/41 1,787, entitled "Method and Apparatus for Cross-Modality 
Comparisons and Correlation", filed September 19, 2002, the contents of which are incorporated 
by reference herein. This application also claims priority under 35 U.S.C. 1 19(e) to U.S. 
Provisional Application Serial No. 60/425,288, entitled "Method and Apparatus for Comparing 
and Correlating PET and X-ray Images", filed November 12, 2002, the contents of which are 
incorporated by reference herein. 

Field of the Invention 

[0002] The present invention relates to a method and an apparatus for determining a 

biopsy location in a body part, and more particularly a method and an apparatus for correlating 
image data obtained from at least two separate devices to determine a biopsy location in a body 
part. 
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Description of the Related Art 

[0003] Increasing the number of medical imaging studies that apply to a single feature or 

to several features can increase the diagnostic confidence of the physician interpreting the 
studies. Diagnostic confidence is increased fiirther if the image sets are correlated; i.e., the 
spatial coordinate systems of the image sets are identical. For display purposes, once the spatial 
coordinate systems are shared, it is often helpfiil to display the images in a single window. In the 
past, such "correlative image displays" have been implemented by using gray-scale for one 
image set (i.e., x-ray) and a color scale for the second set. Altematively, one image set uses hue 
and the other intensity. Aside fi*om increasing diagnostic confidence, correlating images can be 
usefiil if each image set has a different intrinsic utility. For example, an imaging modality such 
as x-ray imaging has high spatial resolution and is therefore often better for guiding 
interventions, because the spatial resolution allows the user to avoid important anatomic 
structures of interest (e.g., major blood vessels). Another imaging modality (e.g., positron 
emission tomography, or "PET") is usefiil for providing biochemical and/or physiological 
information about structures in the human body. 

[0004] It is known in the literature that PET and x-ray images can be combined. For 

example, see 1. Weinberg et al, "Combining X-Ray and Functional Mammography Images", 
Radiology 1997, pp. 205-261. As another example, see I. Weinberg et al., "Implementing PET- 
Guided Biopsy: Integrating Functional Imaging Data with Digital X-Ray Mammography 
Cameras", Proceedings of SPIE Volume 4319, Medical Imaging 2001: Visualization, Display, 
and Image-Guided Procedures, published May 2001. 
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SUMMARY OF THE INVENTION 
[0005] In one aspect, the invention provides a system for determining a biopsy location 

in a body part. The system includes a first device configured to obtain digital physiological 
image data about the body part, a second device configured to obtain second image data about 
the body part, a monitor configured to display the second image data, a signal processing module 
that includes an analog-to-digital converter configured to digitize the second image data, a 
memory configured to store the digital physiological image data and the digitized second image 
data, and a correlator coupled to the memory and configured to correlate the digital physiological 
image data with the digitized second image data and to produce a combined image as a result of 
the correlation. A determination of a biopsy location is made on the basis of the combined 
image, or on the basis of features derived firom the two images. The first device may include a 
positron emission tomography scanner machine. The second device may include one of the 
group consisting of a digital x-ray machine, an x-ray mammography machine, an x-ray cranial 
axial tomography machine, a magnetic resonance imaging machine, and an ultrasound machine. 
The system may also include a localization device configured to select a preferred subset of the 
second image data based on the digital physiological image data obtained fi"om the first device. 
The locaUzation device may include a computer mouse. The first device may be configured to 
use a predetermined spatial coordinate system. The correlator may include a transformer 
configured to transform at least one of the digital physiological image data and the digitized 
second image data into the predetermined spatial coordinate system. 

[0006] In another aspect, the invention provides a method for determining a biopsy 

location in a body part. The method includes the steps of obtaining physiological image data 
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about the body part, obtaining independent second image data about the body part, correlating 
the second image data with the physiological image data, producing a combined set of image 
data based on the correlating, and determining a biopsy location based on the combined set of 
image data. The second image data may include anatomical image data, and the step of 
obtaining second image data may be performed by using one of the group consisting of a digital 
x-ray machine, an x-ray manmiography machine, an x-ray cranial axial tomography machine, a 
magnetic resonance imaging machine, and an ultrasound machine. The step of obtaining 
physiological image data may be performed by using a positron emission tomography scanner 
machine. The obtained physiological image data may be in digital form, and the method may 
further include the step of digitizing the obtained second image data. The method may further 
include the step of selecting a preferred subset of the obtained second image data based on the 
obtained digital physiological image data. The step of selecting a preferred subset may be 
performed by using a computer mouse. 

[0007] In yet another aspect, the invention provides an apparatus and a method for 

coupling a non-networked device to a computer network by capturing an output signal firom the 
non-networked device, digitizing the captured signal, and processing the digitized signal with a 
computer for presentation and transmission over the computer network. The non-networked 
device may include a monitor that is configured to display the output signal to be captured. In an 
alternative embodiment, the invention provides an apparatus and a method for coupling a first 
device to a second device by capturing an output signal fi-om the first device, digitizing the 
captured signal, and sending the digitized signal to the second device. The first device may 
include a monitor that is configured to display the output signal fi-om the first device. 
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BRIEF DESCRff TION OF THE DRAWINGS 
[0008] Figure 1 is a block diagram of an apparatus for correlating image data according 

to a preferred embodiment of the present invention. 

[0009J Figure 2 is a flow chart that illustrates a method of correlating image data 

according to a preferred embodiment of the present invention. 

DETAILED DESCRIPTION OF THE INVENTIQN 
[0010] The present invention is a method and apparatus for obtaining x-ray or image sets 

for correlation with a physiological imaging set. The invention allows a device used for 
physiological imaging (hereinafter referred to as a "first device") to "grab" images from any of a 
variety of other devices (hereinafter refeired to a "second device"), without substantially 
modifying the miderlying software or hardware processes in any such second device. This 
feature is very important, because substantial modification of medical devices may affect the 
validation of such devices by medical regulations (e.g., according to the U.S. Food and Drug 
Administration). An example of a first device is a positron emission tomography ("PET") 
scanner machine. Examples of second devices include digital x-ray machines, x-ray 
mammography machines, x-ray cranial axial tomography (CT) machines, magnetic resonance 
imaging (MRI) machines, ultrasound machines, or any other medical imaging device that 
provides an image to a computer monitor. 

5 

Doc #:WASOI (2I5535-(X)O45)415O7O69vl;09/12/2O03/Time:n:12 



[001 1] The method involves the use of "frame grabber" circuitry to capture an image 

from an output port of a second device, and then to present that captured image to the first 
device. The frame grabber circuitry includes a signal splitting module the also sends the signal 
containing the captured image to a monitor coupled to the second device. In the process of 
presenting the captured image to the first device, amplification and/or duplication of the output 
signal may be performed, as well as other image or signal processing functions. The frame 
grabber circuitry includes an analog-to-digital converter to convert the monitor output into a 
digital signal that can be manipulated. The captured image from the second device can be 
manipulated via mathematical algorithms (e.g., affine transformations) in software or via digital 
signal processing (e.g., firmware) so as to share a conmion spatial coordinate system with the 
images or data collected with the first device. Altematively, the image from the first device may 
be similarly manipulated to share a common spatial coordinate system with the images or data 
collected with the second device. The manipulated data from the second device can be displayed 
with data from the first device to form a fiised image. 

[0012] An exemplary conventional method of capturing images from a second device 

and sending the captured images to a printer is known in the art, and is used by Codonics, Inc. to 
print images from many imaging devices. In one embodiment, the present invention provides the 
advantage of a method of capturing images from a second device in order to combine image sets 
from the second device with image sets obtained using a first device. The invention includes the 
use of pointing and/or locaHzation devices (e.g., a mouse) which are coupled to the second 
device. Because the images shown on the second device include markers (e.g., cursors) as to the 
position of these pointing devices, the capture of images from the second device (and by 
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reference, the capture of said cursors) represents a feedback loop by which the user can adjust 
the position of the pointing device with respect to features that are evident in either or both of the 
first device image and the second device image to select one or more spatial locations. Thus, a 
location for biopsy can be determined using either an image fi-om the first device or one or more 
combined images fi-om the first device and the second device. The determined biopsy location 
can then be shown to the user of the second device by having the user click or otherwise 
manipulate the mouse of the second device and showing or otherwise signaling the location of 
the second device's mouse cursor with respect to the image fi-om the first device and/or one or 
more of the combined images. 

[0013] Referring to Figure 1 , a block diagram of a preferred embodiment of the invention 

includes a first device 105 and a second device 1 10. The second device 110 includes a localizer 
115, such as a mouse, that can specify locations on images obtained by the second device 110 
(and, with the aid of correlation, on images obtained with the first device 105); a CPU 1 17; a 
signal splitter 120; and a monitor 125. The fi'ame-grabber circuitry 130 is coupled to both the 
first device 105 and the second device 1 10, and includes a data digitizer, such as an analog-to- 
digital converter (ADC). The circuitry 130 may also include a signal amplification fimctionality 
and/or other digital signal processing fimctionalities. The firame-grabber circuitry 130 captures 
the image obtained by the second device 1 10 as displayed on the monitor 125, digitizes the 
captured image, and provides the digitized image to the first device 105. The first device 105 
includes an acquisition section 135 for obtaining an image (e.g., using physiological imaging as 
is commonly obtained with radiotracer imaging); a memory section 140 for holding the digital 
image data corresponding to both devices 105 and 110; and a correlative section 145 for 
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combining the image data and indicating the determined biopsy location to the user. A monitor 
150 may be used to display the combined image data to the user. 

[0014] Referring to Figure 2, a flow chart 200 illustrates a method for determining a 

biopsy location in a body part according to a preferred embodiment of the present invention. At 
the first step 205, image data about the body part is obtained using a first device. Preferably, the 
image data is digital and contains physiological information about the body part. The first device 
may be a positron emission tomography scanner machine. At the second step 210, second image 
data about the body part is obtained using a second device. Preferably, the second image data is 
anatomical image data that is transmittable to a video monitor. The second device may be one of 
a digital x-ray machine, an x-ray mammography machine, an x-ray cranial axial tomography 
machine, a magnetic resonance imaging machine, and an ultrasound machine. At the next step 
215, the video signal fi-om the second image data is captured via digitization (e.g., by an analog- 
to-digital converter), and at step 220, the captured digitized second image data is displayed on a 
monitor. 

[0015] At step 225, a user selects a preferred subset of the captured digitized second 

image data. For example, the user may be able to use a computer mouse to select a specific area 
on the monitor display. Then, at step 230, the preferred subset of the captured digitized image 
data (said data containing anatomical or other information about the body part) is correlated with 
the digital image data fi*om the first device (said data containing physiological information about 
the body part). At step 235, a combined set of image data is produced on the basis of the 
correlation. Finally, at step 240, the user determines a biopsy location based on the combined set 
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of image data. For example, the combined set of image data may be displayed to the user on a 
monitor coupled to the first device, and the user may then make a visual determination of the 
biopsy location. The display of the combined image may also use a spatial coordinate system to 
enable the user to be precise in the determination of the biopsy location. 

[0016] Alternatively, image data from the first and second devices may be presented in 

combination to the user without direct combination of the image data sets. For example, data 
fi"om the second image may be processed with feature extraction software in order to generate 
locations of features of interest that are then superimposed on the first image data display* In one 
exemplary application, the location of a mouse cursor in the second image may be extracted, and 
the extracted location may be displayed as a cursor superimposed on the first image. 

[0017] In another embodiment, a system and a method for coupling a non-networked 

device to a computer network are provided. The system is configured to capture an output signal 
fi^om the non-networked device, digitize the captured signal, and process the digitized signal with 
a computer for presentation and transmission over the computer network. The non-networked 
device may include a monitor that is configured to display the output signal to be captured. An 
exemplary computer network may comprise a picture and archiving and communications system 
(PACS). In an alternative embodiment, a system and a method for coupling a first device to a 
second device are provided. The system is configured to capture an output signal firom the first 
device, digitize the captured signal, and send the digitized signal to the second device. The first 
device may include a monitor that is configured to display the output signal fi-om the first device. 
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[0018] While the present invention has been described with respect to what is presently 

considered to be the preferred embodiment, it is to be understood that the invention is not Hmited 
to the disclosed embodiments. To the contrary, the invention is intended to cover various 
modifications and equivalent arrangements included within the spirit and scope of the appended 
claims. The scope of the following claims is to be accorded the broadest interpretation so as to 
encompass all such modifications and equivalent structures and functions. 

[0019] The contents of each of the following publications are hereby incorporated by 

reference: 

1) 1. Weinberg et al., "Combining X-Ray and Functional Mammography Images'', Radiology 
1997, pp. 205-261. 

2) I. Weinberg et al., "Implementing PET-Guided Biopsy: Integrating Functional Imaging Data 
with Digital X-Ray Mammography Cameras", Proceedings of SPIE Volume 4319, Medical 
Imaging 2001 : Visuahzation, Display, and Image-Guided Procedures, May 2001. 

3) http://www.codonics.com/tech/saindex.htm (undated). 

4) PEM-2400 User Manual, Appendix A to U.S. Provisional Patent Application No. 60/425,288, 
filed November 12, 2002. 

5) PEM-2400 Software Instructions Printout, Appendix B to U.S. Provisional Patent AppUcation 
No. 60/425,288, filed November 12, 2002. 
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